Image Registration Using SIFT and Canonical Correlation Analysis

To improve the quality of SIFT feature matching, a mismatches removal method is proposed based on the collinearity property of the canonical correlation features. The influence of each match on the collineartiy degree is analyzed to find false matches. Firstly, a putative set of matches is obtained based on distances between SIFT feature descriptors. Secondly, the spatial relationship of matched points is encoded by canonical correlation analysis. The match with the largest effect on the collinearity degree is iteratively eliminated. Experimental results show the proposed method can reserve a high rate of correct matches determined by a threshold. Compared with several other mismatches removal methods, it provides comparable or better results. Introduction For the past few decades, image registration is involved widely many applications including image mosaic, change detection, cartography and so on [1]. In feature based image registration methods, feature detection and feature matching are two key steps [2]. Some sophisticated feature detection methods have been applied successfully in image registration [3, 4]. However, since there are usually areas with similar intensity distribution in the reference and sensed images, these local descriptor based matching process will result in mismatches inevitably. To address the issue, Euclidean distance ratio filter [3] is often used to remove unreliable matches in SIFT matching by thresholding the Euclidean distance ratio of closest to second-closest neighbors. However, there are many mismatches if the threshold is high and many correct matches are excluded if the threshold is low. So a convenient mode-seeking algorithm which exploits the scale, orientation, and position information of SIFT features was presented in [5]. More recently, Yan et al. [6] fit a line using the first pair of (kernel) CCA features and remove mismatches by thresholding the distances from points to the line. We will refer the method as CCA-I. The main drawback of these methods is that they tend to obtain higher accuracy at the cost of reserving fewer matches. This leads to the removal of high quality matches and suppresses the registration performance. To reserve more matches, although Random Sample Consensus (RANSAC) [7] is a classic and effective method in image registration [8], its randomness a thorny problem. To reduce the randomness, an Optimized Random Sampling Algorithm (ORSA) was proposed for SIFT based image registration [9]. In this paper, we proposed a mismatches removal method following CCA-I [6] with the aim of detecting as many mismatches as possible while keeping as many correct matches as possible. The method improves CCA-I in two aspects. Firstly, since only the first pair of CCA features is usually not enough to indicate all the mismatches, more CCA features are considered. Secondly, because simple thresholding cannot find all mismatches and the performance also depends on the line fitting scheme in CCA-I, a different but effective collinearity criterion is used to overcome the drawback. International Conference on Materials Engineering and Information Technology Applications (MEITA 2015) © 2015. The authors Published by Atlantis Press 118 A Review of Canonical Correlation Analysis Canonical correlation analysis (CCA) has been a successful method for data analysis in computer vision problems [10]. Given the reference and sensed images, let 1 2 { , , } , n X x x x =  and 1 2 { , , } , n Y y y y =  be the position vectors of one-to-one correspondence pairs obtained from SIFT. Their mean vectors are x μ and y μ . Their within-set covariance matrices are x C and y C . The between-set covariance matrix is denoted by xy C . The aim of CCA is to find two directions u and v , such that the correlation of the two projections ( ),1 T i i x s u x i n μ = − ≤ ≤ and ( ),1 T i i y t v y i n μ = − ≤ ≤ , is maximized. u and v can be obtained from the eigenvalue problems: